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Abstract 



The focus of this paper is on the probability, Ep{0; J), that a set J consisting of a finite 
union of intervals contains no eigenvalues for the finite Gaussian Orthogonal {(3 = 
1) and Gaussian Symplectic (/3 = 4) Ensembles and their respective scaling limits 
both in the bulk and at the edge of the spectrum. We show how these probabilities 
can be expressed in terms of quantities arising in the corresponding unitary {(3 = 2) 
ensembles. Our most explicit new results concern the distribution of the largest 
eigenvalue in each of these ensembles. In the edge scaling limit we show that these 
largest eigenvalue distributions are given in terms of a particular Painleve II function. 
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I. Introduction 



In the standard random matrix models of N x N Hermitian or symmetric matrices 
the probabihty density that the eigenvalues lie in infinitesimal intervals about the 
points xi, - ■ ■ ,xn is given by 

Pp{x,, ■■■,xn) = Cnp e-^S Yl \x, - Xkf, 

j<k 

where the constant Cnp is such that the integral of the right side equals 1. In the 
Gaussian ensembles the potential V{x) equals and the cases /3 = 1, 2 and 4 
correspond to the orthogonal, unitary, and symplectic ensembles, respectively, since 
the underlying probability distributions are invariant under these groups. 

When j3 = 2 the polynomials orthogonal with respesct to the weight function 
g-2V{x) play an important role. If (pi{x) {i = 0, 1 ■ ■ ■) is the family of functions 
obtained by ort honor malizing the sequence x^ e~'^^^\ then 

P2{xi, - ■ ■ ,xn) = det{KN{xi,Xj)) = !,■■■ ,N), 

where 

N-l 

KN{x,y) := J2 Vi{x)Vt{y)- (1) 
It follows from this that the so-called "n-level correlation function" is given by 

Rn2{Xi,- ■ ■ ,Xn) = det{KN{Xi,Xj)) (z, j = 1, ■ ■ ■ ,n), (2) 

and the "n-level cluster function" by 

Tn2{Xi, ■■■ ,Xn) = ^KN{Xal,Xa2) " ■ ■ KN{Xa{n-l), X^n) (3) 

where the sum is taken over all cyclic permutations a of the integers l,---,n, in 
some order. (See [|,(5.1.2, 3) and (5.2.14, 15).) The probability ^2(0; J) that no 
eigenvalues lie in the set J is equal to the Fredholm determinant of the integral 
operator on J (more precisely, on functions on J) with kernel Ki^{x,y). There are 
analogues of this for scaled limits of these ensembles. If one takes a scaled limit 
in "the bulk" of the spectrum for the Gaussian unitary ensemble then (^ and (^) 
become 

Rn2{xi,- ■ ■ ,Xn) = det{S{xi,Xj)) (z,j = 1,- ■■,n), (4) 

Tn2(yXi, ■ ■ ■ , Xn) = ^ S{Xai, Xa2) ' " " 5'(X(j( 

n~l) ) X(j-n 

), (5) 

where 

1 sin(x - y) 

S{x,y) := . 

TT X — y 
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Now -^2(0, J) is the Fredholm determinant of the operator on J with kernel S{x,y). 
If one scales the same ensemble at "the edge" of the spectrum this is replaced by the 
"Airy kernel" 

Ai{x)Ai{y) - Ai{x)Ai{y) 



x-y 

where Ai{x) is the Airy function, and if one scales the Laguerre ensemble (which 

2^ ~ 2' 



corresponds to the choice of potential V{x) = \x — \a\ogx) at the edge one obtains 



the "Bessel kernel" 

'^{x)i){y) -ilj{x)(p{y) 
x-y 



(p{x) = Jaiy/x), il){x) = X(p'{x). 



F. J. Dyson |^ discovered that the introduction of so-called "quaternion determi- 
nants" allows one to write down (3 = 1 and /3 = 4 analogues of (||) and (|^) . We define 
e{x) := \ sgnx and 

sin T C"^ 
S{x) := , DS(x) := S'(x), IS(x) := / S(y) dy, JS(x) := IS(x) - e(x), 

TCX Jo 

(6) 

S{x-y) DS{x-y) 
ai{x,y) := I | , (7) 

JS{x~y) S{x-y) 



(74 (x,?/) : = 



S{2{x-y)) DS{2{x-y)) 



IS{2{x-y)) S{2{x~y)) 

If these 2x2 matrices are thought of as quaternions then the analogues of and (^) 
for /5 = 1 and /5 = 4 are obtained by replacing S{x, y) by api^x, y), and by interpreting 
the right side of as a quaternion determinant. The right side of is a scalar 
quaternion, the scalar being equal to ^ times the trace of the right side when it is 
interpreted as a matrix. It follows from these facts, by an argument to be found in 
A. 7 of 0, that Ejs^O; J) is equal to the square root of the Fredholm determinant of 
the integral operator on J with matrix kernel ai3{x,y). There are similar but more 
complicated matrix kernels for the finite N matrix ensembles ([^, Chaps. 6-7). 

The focus of this paper is £^/3(0; J) for (5 = 1 and 4 when J is a finite union 
of intervals, for both finite and scaled Gaussian ensembles. We shall show how 
these can all be expressed in terms of quantities arising in the corresponding (3 = 2 
ensembles. The -E/3(0; J) are Fredholm determinants of certain matrix- valued kernels, 
and by manipulating these Fredholm determinants we are able to write them as 
ordinary scalar determinants whose order depends only on the ensemble and the 
number of intervals in J, and not on A^ if the ensemble is finite. The entries of 
this determinant contain integrals involving the resolvent kernel for the (3 = 2 kernel 
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(these are the "quantities" alluded to above). For the scaled Gaussian ensembles our 
results for general J are given below in (|19D and (|2TD and for the finite N Gaussian 
ensembles the results for general J are in ( p3D and (|3^). 

The evaluation of the integrals appearing in our above quoted expressions is a 
separate matter. In the cases of greatest interest to us — J a finite interval for the 
scaled ensembles, J a semi-infinite interval for the finite ensembles — there are 
systems of differential equations associated with those integrals. The equations are 
easily solved in the cases of the scaled ensembles and we recover known formulas for 
the probability in these models of the absence of eigenvalues in an interval. (These 
are found in as formulas (6.5.19) and (10.7.5).) It then follows from that all 
these probabilities are expressible in terms of a Painleve function of fifth kind (see 
also 0, 0). For finite N , and J the semi-infinite interval (t, cxd), -E/3(0; J) is the 
probability distribution function Fj:qp{t) for the largest eigenvalue. In we showed 
that when (3 = 2 this is expressible in terms of a Painleve function (this time Piv) 
and we hoped to be able to find representations in the cases [3 = 1 and (3 = 4, also, but 
we were unable to solve the associated system of differential equations. However we 
succeeded for their limits scaled at "the edge " because the equations scale to a system 
which we can solve. To explain our results, we recall that F]\;p{2a\/N + 1) tends to 
the Heaviside function as — >■ oo where a is the standard deviation of the Gaussian 
distribution on the off-diagonal matrix elements |l|]. (Our choice of -Pg(xi, ■ ■ ■ ,Xjq) 
corresponds to a standard deviation a = 1/a/2 which is the usual choice 0. We 
also recall that this result holds for the larger class of so-called Wigner matrices fll], 
but the results that follow are only known for the Gaussian ensembles.) This says, 
roughly, that the largest eigenvalue is within o(l) of 2a^/N, and so this is thought of 
as the right edge of the spectrum. In fact, the largest eigenvalue is within 0(iV~^/^) 
of 2a\/N and we consider here the more refined limits 

as , 



(As the notation suggests, is independent of a.) In earlier work we showed that 
when (3 = 2 this is given in terms of another Painleve function {Pjj — see ( ^2]) below). 
Now we shall find representations for Fi{s) and F^ls) in terms of this same function 
(see (|53D and (|5^) below). The probability densities fpis) = dFj^/ds are graphed in 
Fig. 1. Though these results strictly apply only in the limit as the size of the matrices 
tends to infinity, simulations of finite N GOE, GUE and GSE matrices show that 
the empirical probability density of the largest eigenvalue is well approximated by 
(3 = 1,2,4, for N > 200. 

Although this paper treats exclusively the Gaussian ensembles, the methods ap- 
pear quite general and should apply to other ensembles as well. In particular one 
might expect to be able to express the limiting distribution function for the smallest 
eigenvalue in the (3 = 1 and 4 Laguerre ensembles in terms of a Pm function, as is 
the case for /3 = 2 |T^ . 
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Figure 1: The probability densitities //3(s) = dFp/ds, P = 1, 2, and 4 for the position 
of the largest eigenvalue in the edge scaling limit. The probability density with the 
smaller value of f3 ( "higher temperature" ) has the larger variance. 

In Sections II and III we derive our expressions for Ej^{0; J) for the bulk-scaled 
Gaussian ensembles when J is a finite union of intervals. In Section IV we specialize 
to the case of one interval and see how to recover the formulas cited above for the 
probability of absence of eigenvalues. Section V contains the analogous derivations, 
for general J, for the finite N ensembles. These are more complicated than in the 
scaled case, because the expressions for the analogous matrix functions aNisix, y) have 
"extra" terms. In Section VI we derive the differential equations associated with these 
ensembles when J is a semi-infinite interval, and in Section VII we derive the results 
on the limiting probability distribution for the largest eigenvalues in the orthogonal 
and symplectic ensembles. 

To obtain our formulas for the Fredholm determinants we think of the operators 
with matrix kernel instead as matrices with operator kernels, and then manipulate 
their determinants (2x2 determinants with operator entries) in a way the reader 
might find suspect. These manipulations are, however, quite correct and in the last 
section we present in detail their justification for the bulk-scaled ensembles. For 
the basic definitions and properties of operator determinants we refer the reader to 
Chapter IV of where everything we use will be found. 

There is an alternative route to the results of Section VII, based on the fact that 
the limiting probability distributions are Fredholm determinants involving the scaled 
kernel, the Airy kernel. We did not choose this route because we would have had to 
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give yet another derivation of a set of differential equations for the entries of a scalar 
determinant, and because we would have had to present yet another justification for 
the manipulation of the Fredholm determinants, in this case involving the Airy kernel. 
This would be more delicate than the justification in Section VIII for the sine kernel 
because the Airy kernel is not as nicely behaved at — oo. (We do use the Airy kernel 
in our derivation, but only on intervals semi- infinite on the right, where it is very well 
behaved.) 

II. The Scaled Gaussian Orthogonal Ensemble 

First, we introduce some terminology. If K{x,y) is the kernel of an integral 
operator K then we shall speak interchangeably of the determinant for K{x, y) or 
K, and the determinant of the operator I — K. We denote by S and e the integral 
operators on the entire real line R with kernels S{x — y) and e{x — y), respectively (see 
d^)), and write D for d/dx. Notice that DS has kernel DS{x — y) (fortunately!) and it 
follows from the evenness of S{x) that IS{x — y) and JS{x — y) are the kernels of the 
operators eS and eS — e, respectively. We denote by X the operator of multiplication 
by Xj{x), the characteristic function of J. 

Ei{0] JY equals the determinant for the kernel (Ji{x, y) on J, or, what is the same 
thing, the determinant for the operator with kernel 

/ S{x-y) DS{x-y) \ 

Xj{y) 

\ JS{x~y) S{x-y) / 
on R. This can be represented as the 2x2 operator matrix 

/ XSX XDSX 

V X{eS - e)X XSX 
Since De = I, the identity operator, this can be factored as 

/ XD \ / eSX SX 



(9) 



V X J \ {eS - e)X SX 



(10) 



It is a general fact that for operators A and B the determinants for AB and BA 
are equal. So we can take the factor on the left above and bring it around to the 
right, combine the two factors X into one, and find that the above can be replaced by 



eSXD SX 
{eS-e)XD SX 



(11) 
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Subtracting row 1 from row 2 and then adding column 2 to column 1 we see that the 
determinant for this is the same as that for 



/ eSXD + SX SX\ 

^ -eXD J 
and so equals the determinant of the operator 

/I- eSXD - SX -SX \ 



(12) 



V 



eXD 



I 



Next we subtract column 2, right-multiplied by eXD^ from column 1 and then add 
row 2, left-multiplied by SX^ to row 1. (Column operations are always associated 
with right- multiplication and row operations with left.) The result is 

II- eSXD -SX + SXeXD \ 

V /J' 

and the determinant of this equals 

<\ei{I -eSXD-SX + SXsXD). (13) 

So we have shown that £^i(0; J)^ equals the determinant of the operator with 
scalar kernel 

/ - eSXD -SX + SXeXD -SX-S{I - X)sXD, 

where we used here the fact that e and S commute. Now / — -S'X is precisely the 
operator which arises in the bulk-scaled Gaussian unitary ensemble. (In particular 
its determinant, which is the same as the determinant of I — XSX, is exactly E2{0; J).) 
Because J is a finite union of intervals the last summand will turn out to be a finite 
rank operator, so if we factor out / — SX, whose determinant we know, then we 
obtain an operator of the form I— finite rank operator, whose determinant is just 
a numerical determinant. Of course this factoring out requires introduction of the 
inverse {I — SX)~^ which is why the resolvent kernel for SX appears in the final result. 
Now for the details. We denote by R the resolvent operator for SX, so that 

(I - SX)~^ ^ I + R, 

and by R{x, y) its kernel. Observe that this is smooth in x but discontinuous in y. 
Factoring out / — SX and using the fact that determinants multiply, we obtain 



Ei(0; Jf = ^2(0; J) det (I - (S + RS){1 - X)eXD). 



(14) 
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To write the last operator more explicitly we denote the intervals comprising J by 
(a2fc_i, a2fc), (fc = 1, ■ ■ ■ , m), and set 

ek{x) := e{x - ak), 6k{y) := S{y - ak), Rk{x) := R{x,ak), 
where the last really means 

lim R(x, y). 

y&J 

We use the notation a® (3 for the operator with kernel a{x) l3{y), the most general 
finite rank operator being a sum of these. For any operator A we have 

A{a(g)(3) = {Aa®(3), {a (g) l3)A = {a ^ A'/3), (15) 

where A* is the transpose of A. 

It is easy to see that the commutator [X, D] has the representation 

2m 

[X,D] = Y.{-l)%®Sk, 

k=l 

and so 

2m 

e[X,D] = Y.{-l)''ek®Sk. (16) 
k=i 

Since eD = / we have 

{l-X)eXD = {l-X)s[X,D], 
and we deduce the representation 

2m 

(5* + RS){1 - X)eXD = ^(-1)*^(5 + RS){1 - X)ek ® Sk. 

k=l 

The determinant for this is evaluated using the general formula 

n 

det(/-^afc®/?fc) = detfe,fc-(aj,/?fc)) , (17) 

where {aj,f3k) denotes the inner product. (This is an exercise in linear algebra. Its 
generalization to an arbitrary trace class operator is in |0].) Those that arise in our 
case are 

{{S + RS){1 - X)e„ 6k) = ((1 - X)e„ {S + S R')6k) = ((1 - X)e„ Rk), (18) 

where we have used the fact that {S + S R*)X = R. (This is perhaps most quickly 
seen by writing R = ET=iiSX)'' and using S = S**.) Thus we have established the 
formula 

E,{0; Jf = ^2(0; J) det - (-1)^(1 - X)e,,Rk)) . ^ . (19) 
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III. The Scaled Gaussian Symplectic Ensemble 

Because of the factor 2 in the arguments of the functions in the matrix (P) we 
make the variable changes x — > x/2, y y/2, and find that E^i^O; J/2) is the square 
root of the determinant for operator with kernel 

^ / S{x-y) DS{x-y) \ 

^M^) My)- 

\IS{x-y) S{x-y) J 
Thus the operator matrix (^) of the last section is replaced by 

^ / XSX XDSX 

^ V XeSX XSX 
Proceeding exactly as before we find that ([1^) is replaced by 

, / eSXD + SX SX 



V 



) 



and ([ID by 

det {I -^SeXD ~^SX). 
But notice that the operator here equals 

I - SX-^Se[X,D]. 



Using this we factor out / — SX as before and conclude that the analogue of p^ ) is 
^4(0; J/2 f = ^2(0; J) det " ^(^ + R)Se[X, D]). (20) 
Using (|TB|) once again we see that 

2m 

{S + RS)6[X, D] = J2i-^)HS + RS)6k 5k, 

k=l 



and we obtain the analogue of (p^Qf), 

^4(0; J/2f = E^iO; J) det (j,,, - ^^^^ (21) 



where we used the fact that in the inner product we may use the identity 6k = x^k 
since all evaluations are done by taking the limit from within the interval. 
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IV. The Case J = {-t,t) 

In this case we have m = 1, ai = —t, 02 = t, and 

ei{x) = e{x + t), e2{x) = e{x — t), Ri{x) = R{x, —t), R2{x) = R{x, t). 
If we define 



/oo 
R{x, t) e{x =F t) dx 
-00 



then 

- (£1, = (£2, R2) = X+, -(£2, Ri) = (eu R2) = I-. (22) 

The first equalities above used the evenness of R (i.e., the fact R{—x,y) = R{x, —y)) 
and the oddness of e. 

To evaluate these integrals we find expressions for their derivatives with respect 
to t. Observe that -R(x, y) depends on the interval J as well as x and y. Formulas 
(2.9) and (2.22) of ^ give, for y e J, (see also Lemma 2 in [|n[]) 

^i?(x, y) = R{x, -t) R{-t, y) + R{x, t) R{t, y), 

+ ^) ^(^' y) = ^(^' ^(-^' y) - ^(^' ^(^' y)' 

from which it follows that 

d d 

— i?(x, t) =2 R{x, -t) R{-t, t) - Q^Rix, t). 

Hence, applying the product formula to the integrand and using de{x ^ t)/dt = 
T^{x =F ^)) we find that 



— J± = / [2 R{x, -t) R{-t, t) - ^R{x, t)] e{x ^t)dxT R{±t, t)- 
dt J —00 ux 



Integrating by parts the term involving dR{x, t) / dx and using parity give 

= -2 R(-t, t) + R(±t, t) T Ri±t, t). 

dt 

Adding these two identities gives for X+ + X__ the differential equation 

^{X+ + J_) = -2 R{-t, t) (J+ + J_) + 2 R{-t, t), 
dt 

whose general solution is 
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(The function R in the integral is the resolvent kernel for the interval (— r, r).) Since 
Xj-(O) = we have c = — 1 and so 



Similarly if we subtract the two identities for dX±/dt we obtain 
d 



(23) 



whose solution is 



-(J+ - J_) = 2 i?(-t, t) (J+ - J_) - 2 i?(-t, t), 



(24) 

Because of (^2]) these relations determine the inner products {ej,Rk) which arise 
in (^). For the determinant in (|19D we need the inner products ((1 — 'X.)ej, Rk). But 
observe that 



((1-X)£i,i?2) = ((1-X)£2,i?2) = ^ / R{x,t)sgnxdx = ^±^ 



1 _ e-2/o^(--'-)'^- 



((1-X)£i,i?i) = ((1-X)£2,i?l) 



J++X- 



-2 /J iJ(-r,r) dT _ 



2 2 

So we have all the inner products we need. 

It follows from the last displayed formulas that the determinant on the right side 
of (|T^) equals 



1+ e JO 1— e -JO 



, -2 f*fl(-T,r)dr -2 f' fl(-r,T)dr 

1— e -^0 1+ e -^0 



-2 /J H(-r,r) dr 



(25) 



Now in this case J = (— t, t) we have 

d 



dt 



\ogE2iO;J) = -2R{t,t), 



and so 



^2(0; J) = e-2/o^(-'-)<^-. 
Thus from ([T9|) and this fact we deduce 

Ei(0; J) = e-/o[«(-'-)+«(-'-)l'^-. 



(26) 



(27) 
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For the symplectic ensemble in this special case we use (|22D, 



and 



to 



evaluate the inner products appearing in the determinant on the right side of (|2T|) 
and find that it equals 



1 



1+ 


J- 


2 


2 


2- 1 




2 


2 




- J_ 

2 



X4 



(28) 



by (H) and (H. Hence, from (|TD and (|2D, 

^ ^ - /o[i?(r,r)+i?(-r,r)]dr , - /J [fl(r,r)-fl(-r,r)] dr 



£^4(0; J/2) 



(29) 



To see that ( pTj ) and ( [29| ) are just the known formulas (6.5.19) and (10.7.5) of 
we recall the notations 

00 00 00 

Z)(t):=n(l-Ai), Z^+(t):=n(l-A2^), /^-(t):=n(l-Wi), 



i=0 



i=0 



i=0 



where Aq > Ai > ■ • ■ are the eigenvalues of the kernel S{x — y) on {—t,t). Of course 
D(t) is the determinant for S{x — y) while D±{t) are the determinants for the kernels 

S±{x,y) := ^(5(a;-y)±5(a; + ?/)). 
The resolvent kernels for these are 



R±{x, y) = -{R{x, y) ± R{x, -y)) 
and from these we obtain the integral representations 



(30) 



analogous to (Pq). Hence (p^) and (^91) are (6.5.19) and (10.7.5), respectively, of 
We mention here that the left side of (El) equals 



and so in view of 



— j R{x,t)dx 
the formula can be rewritten 



12 



This is equivalent to identity (A. 16.6) of and, although this might not be obvious 
on first comparing the two, our derivation of it has elements in common with the 
derivation, attributed to M. Gaudin, given in 0. 

Finally, we mention that (^) and (|30| ) may be expressed in terms of Painleve V 
following p|. (See |Tl| for a simplified treatment.) 



V. The Finite N Gaussian Ensembles 

The analogue of the matrix kernel (0) for the finite GOE is given by (6.3.8) 
of 0] and so we can write down the analogue of (|^). We shall denote here by 5" 
the operator with kernel KN{x,y) given by (|I]). Recall that the determinant for this 
operator on J equals .^2(0; J) for the finite N GUE. We also write 

^(x) = i^y/' ¥.^(x), ^(x) = i^y/' v.^_i(x). 

Then for even (this case is slightly simpler) the analogue of (^) is 

/ S + 'ijj(»eip SD -t/j^ip 



X 



\ eS — e + eifj ® eip S + eip^ip 



X. (31) 



(See [0, Sec. 6.3.) Now we have to be careful because 5" does not commute with e 
and D. But we have the simple relations 

[S, D] = ip(^tlj + (»(p, [e, S] = -Eip ®eilj-etlj(^ e^p. (32) 

The first follows from (2.20) of fl^ (where finite A^ GUE is called the "Hermite 
ensemble" ) , and the second follows from the first upon left- and right-multiplying by 
e. (We used ([T5|) here and the antisymmetry of e. We shall make similar use of this 
below.) We write the first row of the matrix in (^TI) as 



D {eS + eip(g) Eip, eSD - eip (g) (p) = D {Se - etp (g) eijj, S + eip (g) tjj), 

where we applied (^2]). Using this, and applying (|32D now to the lower left corner, we 
find that the analogue of the second matrix in (|TU[) is 

{Se-eif(g)ei!)X {S + e(f (g i!)X 



( XD \ 

Now we move that matrix \ ^ around to the right and find the analogue of 



{Se - e - eip ® e%l))X {S + e^p (g %Ij)X 

KD 
^0 X ^ 
the right side of (|TT|) , 

/ {Se -eifg) ei))XD {S + eip® i))X 
\ {Se - e ~ Sip (g ei))XD {S + eif ® i))X 
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The same row and column operations as before reduce this to 

{Se -Eif® ei))XD + {S + eif ® i))X {S + eip® i))X 

-eXD 



the analogue of (p!2D. We complete the computation just as before and find that the 
determinant for this equals the determinant of the scalar operator 

/ - SeXD + {ef eij)XD + {S + ® i))X{eXD - I) 

= I - SX - S{1 - X)eXD -eif^Xi)- (ey? <S) ^) (1 - X)eXD. 

Factoring out / — SX shows that the determinant of the above equals -^2(0, J) times 
the determinant of 

I -{S + RS){1 - X)eXD -Q,®Xi)-Q,®i){l- X)eXD, 

where 

Qe := (/ - Sxy'eip 

and we have used the same notation R as before for the resolvent operator for SX. 
Using (|16D and the general fact (a ® /3) (7 ® ^) = (/?, 7) a (8) 5 we see that the above 
operator equals 

2m 2m 

I-Y.i-^)'iS + RS){l-X)ek05k-Qe®X^-J2i-lfiiP,{l-X)ek)Qe®Sk. (33) 

k=l k=l 

Thus -E'i(0; J)^/i?2(0; J) equals the determinant of this operator. 

The analogue of the matrix kernel (Q) for finite GSE is given by (7.1.5) of 
where now must be odd.Q Because of the factor \/2 in the arguments of the 
functions in the matrix (7.1.5) of 0, we make the change of variables x — > x/\/2, 
y y/V2 and find that ^4(0; J/V2) is the square root of the determinant for 

^ I S + 4)iS)e(f SD -il)®Lp' 
:X\ \X. 



2 



eS + ei}) ® eip S + e^p ^ij) 



Proceedingly analogously to finite GOE leads to the following formula for the 
operator whose determinant is -^4(0; J/v^)^/£'2(0; J): 

1 2m 1 2m 

^ - 9 E(-l)'('^ + RS)ek ® 4 - Qe ® - ^ E(-l)' ^k) Qe ® 4. (34) 
^ k=l ^ fc=l 



-'^In finite = 2?i + 1 GSE the matrices are 2n x In Hermitian matrices with each eigenvalue 
doubly degenerate 0. 
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VI. The Case J = (t, oo) 

Now J has the end-point ai = t and 02 = 00, and we write 6t, Soo, St, ^oo, Rt and 
i?oo for the quantities 5^, e^, i?^ (A; = 1, 2) of the last sections. Note that 

£00 = -^, (1-X)£t = (1 -X)£oo = i?oo = 0. 

With this notation the operators (|33D and (|3^) become 

/ - ® - ^[(5 + - X) + (1 - X)) Qe] ® {6t - (5oo), (35) 

I-Qe®X^ + ]^[{S + RS)et + {ip,et)Qe\®5t + ^[{S + RS)l + {:ilj, l)Qe]®5^, (36) 
respectively. Both operators are of the form 

n 

I-J2ak®Pk (37) 

k=l 

SO to evaluate their determinants using (p!?]) we have many inner products to evaluate. 
We shall introduce several new quantities now and express all the inner products, and 
therefore the determinants, in terms of them. Then we shall write down systems of 
linear differential equations (in the variable t) which in principle determine these 
quantities. 

First, there are 

Q := {I-Sxy^ifi, P := {I-SX)-^iJ, Q,:={I-SX)-hip, P, := {I - Sxy^etfj, 

the third of which we have already met. We use small letters to denote the values of 
these functions at a: = t: 

q = Q{t), p = P{t), qe=Qe{t), Pe = Pe{t). (38) 

(The functions q and p play important roles in the investigation of -£'2(0; J) [|14| and 
we think of them here as known.) 
Next, there are the inner products 

Ue := (Q, Xeif ) = {Qe, Xip), Ve := (Q, XetP) = (P„ X^), 

V, := (P, Xe^) = {Q„ X^j), w, := (P, Xe^) = (P„ X^), 

the last four being analogous to (2.4)-(2.5) of with replaced by e. Our first 
system of differential equations (in which q and p appear as coefficients) will connect 
these with q^ and p^. 
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Finally there are two triples of integrals 

TZi := / R{x,t)dx, Vi := / P{x)dx, Qi / Q{x)dx, 

J —oo J —oo J —oo 

/oo roo poo 

et{x) R{x,t) dx, Vi := / et{x)P{x)dx, Q4 := / et{x)Q{x)dx. 
-00 J —00 J —00 

(The subscripts 1 and 4 indicate that these arise in GOE and GSE, respectively.) We 
shall find systems of differential equations for each of these triples. 



The determinants of ( pq ) and ^3q): 

We consider the GOE operator (^5|) first. If we set 

ai := (^, 1 - X) 

then the operator is of the form (|37D with n = 2 and 



ai = Qe, a2 = ^[{S + RS){l-X) + aiQ,], pi=X^, p2 = 5t-5^. 
We also set 

2 roo 2 

c^p := £^9(00) = - (p{x) dx, := 5-?/'(oo) = - ^p{x) dx. (39) 
2 J —00 2 J —00 

For even a computation gives 

^ ^ ^ (iV/2)! • 

Since = e(p + SX{I — SX)~^eip, we have (^^(cxd) = c^. Therefore, with the notations 
introduced above, 

(ai. Pi) = Ve, (ai, P2) =qe- c^- 

To compute the inner products involving ^2 "we use the fact {S + S R^)X = R, as in 
the derivation of (0), to write 

{{S + RS){1 - X), X^) = {1~X, R^) = {l-X,P-^)=Vi- ai, 

{{S + RS)il - X), (5t) = (1 - X, Rt) = 7^l, 
((5 + RS){1 - X), 5oo) = (1 - X, i?oo) = 0. 
Using these, we find that 

("2, A) = ^(Pi - ai + ai We), (aa, /32) = ^(T^i + ai - ai c^). 
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The GSE operator (|36| ) has the form ( |37D with n = 3. But notice that 
{{S + RS)6t, 5oo) = {st, Roo) = 0, {{S + RS)1, 6^) = (1, i?oo) = 0, 

and (Qe, S^o) = = because for GSE is odd and so ip is an odd function. Thus 
the contribution to the determinant of the last term in (|36|) is and we may discard 
it. The resulting operator is of the form (^) with n = 2 and 

ai = Qe, a2 = -^[{S + RS)6t + a^Qe], (3i = Xij, (^2 = St, 
where we have set 

04 := et). 

The inner products are computed as for GOE above and we find that now 

("2, Pi) = -^("^4 - 04 + a4^Vs), ("2, P2) = -^(^4 + a4qe)- 

Thus the determinants of the operators ( p5D and (^) are expressible in terms of the 
constants Oi, 04 and and the as yet to be determined quantities v^, q^, Pi, TZi, P4, 
and 7^4. The determinants are given by 



(40) 



[l + ln,) + ^q,V,, (41) 



respectively. (The constants oi and 04 drop out, as we see.) 
The first set of differential equations: 



The derivation will not be quite self-contained because we shall refer to ||Tj] for 
some results derived there. First we have the analogues of (2.15)-(2.18) of |M, 

K = -11e, v'e = -(lPe, K = -P(le, w'^ = -PPe, (42) 

which are proved in exactly the same way. (The primes denote d/dt). To derive 



formulas for q'^ and p'^ we use the displayed formula between (2.27) and (2.28) of |T^, 
which in this case gives 

(I + ^ + ^) P{^.y) = -Q{x) ■ {XP){y) - P{x) ■ {XQ){y), (43) 
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where p(x, y) is the kernel of (/ — SX) ^, in other words 6{x — y) + R{x, y). It follows 
from the above that 

(If f d 

l'e = -^J y) ^'^(y) ^y^ ~ J Oy^^^' ^^^^^ dy-q {XP, 6<^) - p {XQ, 6<^). 

The first term on the right side equals 

J p{t,y)v{y) dy = g. 

We treat similarly, and we find that we have derived the equations. 

q'e = Q - - pUs, p'^=p- qwe-pve- (44) 

For the boundary conditions at t = oo, observe that the four functions m^, f^, 
and Wg; all vanish there, whereas 

One of these always vanishes, the first if is odd and the second if is even. 

It is easy to derive a first integral for our system of equations. Using the equations 
we find that 

= V'^{1 - Ve) - w'^Ue + ^^(l - V^) - u'^W^ = -((1 - Ve)il - V^))' - (M^We)', 

and so, since all quantities vanish at oo, 

= 1 - (1 - " Ve) " U^W^. 

The second set of equations: 

It follows from (^) that for y E J 

and so 

^i?(x,t) = -^R{x,t)-pQ{x) -qP{x), (45) 
and from this we obtain 

f* d 

n[=R{t,t)+ / [——R{x,t)~pQ{x)-qP{x)]dx. 

J — oo UX 

This gives our first equation, 

K = -pQi-qVi. (46) 
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By (2.10) of we have 

^ = -R{x,t) g, 

and so ^ 

Q'l = q — q / R{x, t) dx. 

J —OD 

We treat Vi similarly, and so we have our other two equations, 

Q;=g(l-7^l), P^=p(l-7^l). (4?) 

At t = cx) our functions have the values 

7^l(oo) = 0, Qi(oo)=2c^, Pi(oo) = 2c^. (48) 

The system (|46|) , (147|) has a first integral. If we multiply the first equation by TZi 
and use the second we obtain 

7^l7^; = iv[ -p)Qi + {q[ - q)Vi = (QiPO' + n[. 

Integrating gives 

l7^2 = QlPl+7^l. (49) 

(Again the constant of integration is because either c^p or is 0.) Differentiating 
|) and using (|T7| ) again give 

7^l = -p' Qi -q'Vi + 2pg(7^l - 1). (50) 



Solving equations (^) and (jS^) for Qi and Vi and substituting the results into 
we obtain a second-order differential equation for TZi. 

-{\nl-n,) {q'p-p'qf = {pn'i-p'n[+2p'q{n,-l)) {qn'[-q'K + 2q'p{n,-l)). 

To find a differential equation for Vi we use the second relation of (|^) to express IZi 
in terms of Vi, then use ( ^61 ) to express Qi in terms of Pi, and then substitute these 
results into (^9]) . What results is the equation 



The third set of equations: 



To obtain the equations for 7?.4, Q4 and P4 we proceed almost exactly as in the 
last section, replacing the domain of integration (— cx),t) by (— cx),cx)), and inserting 
the factor e{x — t) in the integrands. The only difference is that when we differentiate 
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the integrals we apply the product formula to the integrands. What results is the 
system 

K = -pQi-i'P4, = -g(7^4 + l), v'^ = -p{n^ + i). (5i) 

The solutions of this system are obtained from the solutions of the last by simply 
changing their signs. But notice that the values at t = oo are now given by 

7^4(00) = 0, 24(00) = -c^, 7^4(00) = -c^. 

VII. Scaling GOE and GSE at the Edge 

The goal of this section is the computation of the limiting probability distribution 
functions for the largest eigenvalue in the finite N Gaussian orthogonal and symplectic 
ensembles. The probability distribution functions for finite GOE, GUE and GSE 
are precisely the functions Ei3{0, (t, 00)) of Sec. VI with /? = 1, 2 and 4, respectively. 
We denote them by FAr^(t), and limits we are interested in are 

Ffsis) := lim Fnp{V2N + 



In we showed that the limit exists when j3 = 2 and is given by 

^2(5) = exp I — / {x — s) q{xY dx ) , (52) 



where q is the Pn function determined by the differential equation 

q" = sq + 2q^ 

together with the condition q{s) ~ Ai(s) as s — > 00. (For more details on this solution 
see P, [13.) We shall show here that the limits exist for (3 = 1 and 4 also, and that 

Fi(s)2 = F2(s)e-/r9(-)'^-, (53) 



I /,°° dx , -\ q(x) dx ^ 2 



F,{sH2f = F^is) I ^ ±^ I . (54) 



(Note the similarity to formulas (^Sf ) and (Pq).) 

The reader must see our notational difficulty: q denotes both a Painleve function 
and the function defined by ( ^8]) of the last section. We resolve this difficulty by 
denoting the latter function here by q^ (and denoting the function p of the last 
section by Pn), while retaining the notation q for the Painleve II function. 
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We denote our scaling transformation by r, so that 



We think of s as fixed, and the functions qN, PN,qe, Ue, - ■ ■ ,V4 of the last section as 
being associated with t = t{s). We shall show that these functions of s (perhaps 
after normalization) tend to limits as iV ^ oo and that these limits satisfy systems of 
differential equations which are solvable in terms of the Pu function q. Substituting 
the values of these limits into (|40| ) and ( ^l]) will give (^) and ([5^). Everything will 
be a consequence of the following: 

(i) lim^^oo A^^'/'gjv = liuiM^^ N'^/^pN = q. 

(ii) The limits limTv^ooWe and limAr^oo'^e exist and are equal. 

(iii) The limits of and TZi, ■ ■ ■ , 7^4 all exist. The limits of Qi and Vi differ by a 
constant as do the limits of Q4 and P4. 

(iv) All of the above limits hold uniformly for bounded s. 

These will be established below, but suppose for the moment that they are true. 
Denote the common limit in (ii) by u, the limit of by q and the limits of TZi, ■ ■ ■ , P4 
by:^i,---,P4. 

Let us rewrite the first equations of (^) and (^^ using our present notation and 
letting the prime now denote ^ (= j^^/ \/2N^/^): 

I qN I ^ ( ~ \ 

= 1= -Oe, q=^= — [qN — qN — Pn Up)- 

Taking the limits as ^ 00, using (i) and (ii), gives the system 

«' = -^g, g' = ^(i-2n). 

(The interchange of the limit and the derivative is justified by the uniformity of 
convergence of these derivatives.) We remind the reader that when we apply our 
finite results to GOE and GSE we restrict A^ to even or odd values, respectively, 
and this affects the boundary condition at s = 00. In fact, we have 



u(oo) = 0, q{oo) 



1/v^ in GOE, 



in GSE. 



The reason for the first is that Ue vanishes at 00 and the reason for the second is that 
qe{oc) = Cy,, which vanishes when A" is odd and can be shown to have the limit 
as A^ CX3 through even values. Introduction of 

/•oo 

/X := / q{x) dx 
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as a new independent variable reduces our system of equations for u and q to one 
with constant coefficients wfiich is easily solved. We find that 

u=]^{l-e'^), g = -^e-'^ in GOE, 

u=\{l~\e^-\e-n, q=^{e~^-e^) in GSE. 

Next, we consider the limiting quantities IZi, Qi and Pi. Recalling that these 
arise in GOE, when N is even, we find from (^) that 

:^i(oo) = 0, Qi(oo) = V2, Pi(oo) = 0, 

and so, since by (iii) the last two functions differ by a constant. 

We find now that the limiting form of the system (|i6| ) and (^) is 

^'i = -^(2Pi + v^), v[ = -^{n,-i). 

The same substitution reduces this to a system with constant coefficients, and we 
find the solution to be 

n, = l-e-^, Pi = i=(e-^'-l). 
Similarly, for /3 = 4 when iV is odd we find that 

7^4(oo) = 0, Q4(oo) = 0, P4(oo) = 
that Qi = Vi + -\/2/2, that the system is 

and that the solution is 

If we substitute the limiting values we have found into the formulas (|4y) and (|41|) 
which give the values of the ratios Fiyi{ty / Fmit) and F/v4(t/-\/2)^/F/v2(i) we obtain 
formulas (E^) and (153). 
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It remains to establish our claims (i)-(iv) above. We indicates by a subscript r 
the result of scaling either a function or an operator. Thus, 

Sr := r o S o T, := {p o T, etc.. 

It follows from results on the asymptotics of Hermite polynomials that 

lim N-^/'^ip^{x) = lim N-^/'^i/j^ix) = A{x) (55) 

uniformly for bounded s, where A{x) denotes the Airy function Ai(x), and that there 
are estimates 

N-^^^ifrix) = 0(e-"), N-^/^i/jrix) = 0(e-") (56) 

which hold uniformly in N and for x bounded below. (There is a better bound, in 
which x'^/^ appears in the exponent rather than x, but this one is more than good 
enough for our purposes. See [|10], P- 403.) 

To obtain the scaling limits of 5* and other quantities we shall make use of the 
identity 

S{x,y) = / [(p{x + z) 'ijj{y + z) + 'ijj{x + z) (p{y + z)] dz (57) 



A{x + z) A{y + z) dz 



analogous to formula (4.5) of fl^ 

A{x) A'{y)-A'{x)A{y) 
x-y 



and proved in the same way: The formula on the top of p. 43 of gives 
( d d\ 

( ^ + ^ 1 '^(a;, y) = ^{y) - ^(x) ^{y), 

and the same operator applied to the right side of (ETt) equals 



( d d\ 

Jo \dx^ dy) ^"^^^ ^ + i^ix + z) ip{y + z)] dz 

d 

— [V9(x + z) i){y + z)+ i){x + z) ip{y + z)] dz = -ip{x) ip{y) - ip{x) ip{y). 
oz 



Hence the two sides of (q^) differ by a function oi x — y and this function must vanish 
since both sides tend to as x and y tend to oo independently. 
If an operator L has kernel L{x,y) then Lr has kernel 
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and so from ( |57D we see that the kernel of Sr has the representation 
1 f°° 

Srix, y) = ^^^Q Ivi-rix) + z) ^{T{y) + 2;) + ^{t{x) + z) ^{r^y) + z)] dz, 

and the substitution z z/^/2N^^^ gives 

1 

Sr{x, y) = ^^^^3 [<^r{x + Z) Ipriy + z) + t/j^ix + z) ifriV + z)] dz. (58) 

The asymptotic formulas and the estimates show that 

lim Sr{x,y) = / A{x + z) A{y + z) dz 

N^oc Jq 

pointwise, and in various function space norms as well. This will be very useful. The 
right side, we know, is the Airy kernel. We have been using a bar as notation for a 
limit, so we write the above as 

Srix,y) S{x,y) = A{x + z) A{y + z) dz. (59) 

Jo 

The fact that this converges in the space L2{s,oo) ® L2{s,oo) implies that the 
operator XrSrXr converges in the norm of operators on L2(R) to the operator XSX, 
where X = X(^s,oo)- Since / — XSX is invertible it follows that also 

(/ - XrSrXr)'^ ^{I - XSX)-^ (60) 

in operator norm. The same is true if the space L2(R) is replaced by Li(R), as is 
seen if we use the fact that the norm of an operator XLX on this space is at most 

/ snp\L{x,y)\ dx. 

S y>s 

and that (|59| ) holds with this function space norm as well. 
Let us compute the scaling limit of gAr. We have 

gjV = (/-5X)~V W = ^{t)+SX{I~XSX)-^^ (t) = ^^{s)+SrXr{I-XrSrXr)-^^r (s) . 

It follows from (|55|) and (|56D that N~^/^Lpr A pointwise and in ^2(3,00). The 
second of these facts, together with (pDD, implies that 

(J - XrSrXrY^N-^'^^r ^ (/ " XSXY^ A 

in L2, and then using the fact that ( ^9|) holds in ^2(5, 00) for fixed x, we deduce that 

SrXr{I - XrSrXrY^N-^l^^r ^ SX{I - XSXY^ A 
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pointwise. Putting these together shows that 

hm N-^'^qN = A{s) + SX{I - XSX)-^ A (s) = (/ - XSt)'^ A (s). 

The right side was precisely the definition of q (it transpired that it was a Painleve 
function), so we have shown that 

hm N~^^^qN = q. 

N^oo 

Note that since by (|5^) both ip and ip have the same scahng hmit, the above argument 
apphed to leads to the same result, 

lim N'-'^/^pn = q. 

N^oo 

This gives assertion (i). The uniformity assertion in (iv), for these and the limits 
established below, we leave to the reader. 

Next we consider and v^- Recalling the definition (R^) we write 



e(p (x) = - / (fiy) dy, 

J X 

and so 

POO PCX) 

(e^Ux) = c^- l^^^ v{y) dy = c^- -^=^ dy. 

As — s> oo the constant c^p converges (in fact to 1/ \/2) and the second term converges 
in L^i^s, oo) (in fact to — A{y) dy/ \/2). Thus the function {ei^)r converges in the 
space R + ^2(5, oo). Also, 

Ar-i/6Q^ = - XrS^X^y^^r ^ {I - XSX)-^ A (61) 

in Li(s, 00) n L2{s^ 00). It follows from these limit relations that 

Ue = {Q, Xe<^) = ^^^/g {Qr, {Xe<p)r) 

converges as A^ 00. Notice also that since (f and ip have the same scaling limit, 
(|6T|) holds with Q replaced by P on the left side, from which it follows that has 
the same limiting value as m^. This establishes (ii). 

Finally we come to the quantities TZi, - ■ ■ ,Vi. These are trickier since, although 
our functions scale nicely on (s, 00) for fixed s, they do not scale uniformly on 
(—00, 00). In a sense we have to separate out the parts that get integrated over 
(—00, 00). Beginning with TZi, we use 

R{x,t) = {I - Sxy^S{x,t) = SX{I - XSxy^S{x,t) + S{x,t), 
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and write our integrals over (—00, t) as integrals over (—00, 00) minus integrals over 
(t, 00). Thus 

/oo roo 
SX{I - XSX)-'^S{x, t)dx- SX{I - XSxy^Six, t) dx 
-00 Jt 

/oo /'OO 
S{x,t)dx— / S{x,t)dx 
-00 Jt 

1 1 
SyXyI^-I- Xq-S-j-X^^ S-j-(^x^ dx / S^Xq-(^l XfS^X^^ S^-i^x^ dx 



/oo /"OO 
Sr{x,s)dx— / S't-(x, s) dx. (62) 
-00 J s 

(The factors l/v^iV^/^ arising from the variable change are incorporated in the ex- 
pression for Sr{x,s).) We think of the first integral on the right side of ( |62l) as the 
inner product of the functions 

/oo 
ST-{x,-)dx and {I — XSrXr)~^Sr{-, s). (63) 
-00 

on (s, 00). Now it follows from ( |58D that 

/oo ]^ / /"OO /"OO \ 

^ 5^(X, y) C/X = ^^^^g (^C^ V^r(^) + ¥'r(^) C?^j , (64) 

which converges in L2{s, oo) as ^ oo. Also Sr{-, s) converges in ^2(5, 00), so the 
same is true of the second function in (|63D. Thus the inner product itself converges, 
and this shows that the first term on the right side of (^) converges. The second term 
(p2| ) is treated in much the same way — the only difference is that the first integral in 
(|63|) is taken over (s, 00). The next term on the right side of (|6^) is simple — to show 
that it converges requires only the representation (^^ with y = s — and the last term 
is analogous to this one. 

Turning to Qi, we write it similarly as 



/•oo rOO POO 

SX{I -XSxy^ip{x)dx- SX{I -XSxy^ip{x)dx+ ip{x)dx- ip{x) 

Jt J — CO Jt 



dx 



J SrXr{I — XrSrXr) /o/Yl/e ^'^ ~ j SrXr{I — Xr SrXr^ 



dx 



We treat the integrals just as we did those for 7^l, with ip^/y/2N'^/^ repl acing St(-, s), 
to show that they have limits as 00 and, of course, also has a limit. The 

quantity Vi is handled similarly, with ip replacing ip evrywhere. That the limits of 
Vi and Qi differ by a constant follows immediately from the fact that and have 
the same scaling limit on (s, 00). The discussion of the quantities 7^4, Q4 and V4, is 
entirely analogous, and (iii) is established. 
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VIII. Justification for the Determinant Manipulations 



In this final section we give the rigorous justification for the determinant manip- 
ulations in Sections II and III. We begin with GSE, which is slightly easier. The 
quantity of interest is the determinant for the operator on L2{J) with kernel 



/ S{x — y) DS{x — y) 
\IS{x-y) S{x-y) 



where 

S{x) = DS{x) = S'{x), IS{x) = / S{y) dy. 

Tlx Jo 

This kernel is smooth and so the operator is trace class. Our determinant is the same 
as that for the operator on L2(R) 



XSX XDSX 
XISX XSX 



(65) 



where S, DS, IS are the operators with corresponding difference kernels and X is 
multiplication by Xj. We shall show that we can remove all the operators X which 
appear on the left, if we interpret S, DS, and IS as acting between appropriate spaces. 
Recall that the Sobolev space Hi is given by 

{/ G -L2 : / is absolutely continuous and /' G L2}, 

with 

II / \\hi= \/|| / Ilia + II /' Ilia- 

The mapping 

f^f + Df {D = -^) (66) 
is an isometry from Hi onto L2. 

Lemma 1. SX and DSX are trace class operators from L2 to Hi. 
Proof. If the kernel K{x, y) of an operator K satisfies 

/ / \K{x,y)^ dy dx < 00, [ [ \— K{x,y)\'^ dy dx < 00, 
Ju J J Jn J J oy 

then KXj is trace class for any set J of finite measure. (See, for example, pp. 118-119 
of 0.) It is immediate that D^SX : L2 ^ L2 is trace class for each n > 0. If we 
recall the isometry (|66|) we see that to show that D'^SX : L2 ^ Hi is trace class it is 
enough to show that (J + D)D"'SX : L2 ^ L2 is. But we know this. 
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Next we enlarge our spaces L2 and Hi by adjoining a single function to each: 



L2:={f + ce:fe L,, c G C}, := {f + cIS : f e H^, c G C}, 
which are Hilbert spaces when endowed with the norms 



respectively. 

Lemma 2. ISX is a trace class operator from L2 to Hi. 

Proof. Integration by parts on the constituent intervals of J shows that for all f E L2 

{IS){f){x) = Y.{-1)' IS{x - a,) {If){a,) + J S{x - y) {I f){y) Xjiy) dy, (67) 

where := Jq f{y) dy. The map / (If) - Xj is bounded from L2 to L2 and so 

by Lemma 1 the integral on the right represents a trace class operator from L2 to Hi. 
The maps / — > [I f){ai) are continuous linear functionals on L2 and each function 
IS{x — ai) belongs to Hi since 

IS{x - ai) - IS{x) G Hi. 

So the sum on the right side represents a finite rank, and hence trace class, operator 
from L2 to Hi. This completes the proof. 

Let us consider the operator represented by the matrix 



1 



/ SX DSX \ 

(6^ 

\ ISX SX J 



which is the same as ( pSQ except that all factors X that appeared on the left were 
removed. Since X : L2 ^ L2 is bounded, it follows from Lemmas 1 and 2 that this 
is a trace class operator from L2 © L2 to itself. (Actually, of course, the operator is 
trace class from L2 © L2 to Hi® Hi, but we don't use this.) 

We use now, and several times below, the fact that the determinant for an operator 
product AB is the same as for BA as long as one of the two factors is trace class 
and the other is bounded. They do not have to act on the same Hilbert space; one 
operator can map a space H to a space H' as long as the other maps H' to H. The 
two products then act on the different spaces H and H'. 

Lemma 3. The determinants for the operator (|65D on L2 © L2 and the operator (|68|) 
on L2 © L2 are equal. 
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Proof. Because X is idempotent we can insert a factor 



on the right side 



X 
^ X ^ 

of (p8| ) without changing it. We can bring this factor aound to the other side and 
deduce that the determinant for (|6^) as an operator on L2 © L2 is the same as that 
for ( |65D as an operator on L2 ® L2. But the range of this operator is contained in 
L2 (B L2, so the determinant for it is the same when considered an operator on this 
space. 



It follows from the lemma that we can replace 
can write it as the product 



by (H). Since D{IS) = S we 




It follows from Lemmas 1 and 2 that the factor on the right is trace class from L2 ©L2 
to Hi © Hi while the factor on the left is bounded from Hi © Hi to L2 © L2 since 
D : Hi ^ L2 is bounded. It follows that the determinant for (|68|) on L2 © L2 is the 
same as the determinant for 



/ ISX SX \ / D 
^ISX SX ) \ I 



/ ISXD SX 
V ISXD SX 



on Hi® Hi. The matrix entries are operators on the same space, Hi, so that we 
can perform what amounts to row and column operations on them. (This was not an 

' / ^ 



accident!) Multiplying on the left by the matrix 



mverse, 



and on the right by its 




we see that the last determinant is the same as that for 




and so it is the determinant of 



(I- \ISXD - \SX -\SX 



\ 







/ / -\sx 



\ISXD -\SX \ 







1 1 



Both factors on the right are operators of the form /+ trace class operator on Hi® Hi, 
and the factor on the left is of the form /+ nilpotent operator and so has determinant 
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1. Hence the determinant of the product equals the determinant of the second factor, 
which equals 

/ - -ISXD - -SX 
2 2 

We rewrite the operator as 

I ~ SX-^{ISXD- SX). (69) 

A variant of (|6^) is 

{ISX){f){x) = ^{-l)' IS{x - a,) f{a,) + J S{x-y)f{y)xM dy, 
which gives 

ISXD-SX = Y.{IS)k®6k, 

where {IS)k{x) := IS{x — a^) and 6k{x) := 6{x — a^). The tensor product denotes 
the operator which takes a function in Hi, evaluates it at and multiplies this by 
{IS)k- A similar interpretation holds for any tensor product u® v where u belongs 
to a Hilbert space and v to its dual space. 

We extend the domain of 5* to all of L2 by defining 

{Sf){x) = J S{x-y)f{y) dy, 

the integral being conditionally convergent. It is easy to see that S : L2 Hi and 
{IS)k = Ssk, where ek{x) := e{x — a^). Thus we can write ( |69D as 

I-SX-^J2i-^)'Ssk0 5k. (70) 

Recall that our operators act on Hi. Now / — SX is invertible as an operator on 
this space as well as on L2, since the eigenfunctions of SX belonging to nonzero 
eigenvalues are the same for the two spaces. For the same reason both interpretations 
of the operator give the same value for the determinant. We denote the inverse of the 
operator, as before, hy I + R. Factoring out / — SX shows that the determinant of 
([Top equals £^2(0, J) times the determinant of 

i-lT.i-^)'is + RS)^k^^'^- (71) 

Recall the definition Rk{x) := R{x,ak). 

Proposition. The determinant of (|7Tp on Hi equals the determinant of 
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on L2. 

Proof. We use inner product notation [u, v) to denote the action of a dual vector v on 
a vector u. The determinants of (|7I|) and ( |T2D are scalar determinants whose entries 
contain the inner products 

{{S + RS)e,,5j,), {e„Rk), 

respectively, in position (j, k). We shall show that these are equal. 

We begin with the observation that R^X = XR when these are thought of as acting 
on L2. This is so because R is the resolvent operator for SX and S is symmetric. It 
follows from this that if / and g belong to L2, with g supported in J, then 

{{S + RS)f, g) = if, (SX + SXR)g) = (/, Rg), 

the last by the resolvent identity. Suppose h has integral 1 and is compactly supported 
(in if k is odd, in R~ if k is even), set g{x) = n h{n{x — 0^)) and let n 00. We 
obtain 

iiS + RS)f,5k) = if,Rk). 
Replace / by /„ := ejX[-n,n] and let n — 00. Since 

{S + RS)fn (5 + RS)ej 

uniformly on compact sets we deduce the desired identity 

{{S + RS)e,,6u) = {ej,R,). 

Thus ( pTD is completely proved. 

We now turn to GOE. The reason this is slightly awkward is that the operator K 
on L2{J) with kernel 



S{x — y) DS{x — y) 

\ IS{x-y)-e{x-y) S{x-y) 



(73) 



is not trace class . So its classical Fredholm determinant, which is what we want, 
is not given by det(/ — K), which is not defined. It is instead given in terms of its 
regularized determinant det2 by the formula 



det2(/ - K) e 



-trK 



where tr K denotes the sum of the integrals of the diagonal entries of the kernel of 
K. Rather than deal with regularized determinants we approximate the kernel by a 
smooth kernel, evaluate the resulting Fredholm determinant, and pass to the limit at 
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the end. Thus we replace the term e{x — y) in (|73D by ?7„(x) := ri{n{x — y)), where 
7] is a smooth function which equals e outside a finite interval. Then the resulting 
operator Kn is trace class, and det2(/ — K) e~^^^ is equal to the limit of det(/ — Kn) 
as n ^ oo. 

Proceeding as before, we find that det(/ — Kn) is equal to the determinant of 

I-ISXD-SX -SX 

VnXD I 

on Hi(B Hi, where rjn denotes convolution by rin{x). The operator can be factored as 
/ / -SX \ f I- ISXD -SX + SXr]nXD \ / / \ 



V VnXD I J 



all factors being of the form /+ trace class operator on Hi® Hi, and so its determinant 
equals the determinant of the operator 

I - SX- ISXD + SXr]nXD 

on Hi. But Xr]nX — » XeX in the norm for operators on L2 (where e denotes convolution 
by £{x)) and it follows from this and the second part of Lemma 1 that the above 
operator converges to 

I-SX- ISXD + SXeXD (74) 

in the trace norm for operators on Hi. Consequently it is the determinant of this 
which will be our final answer. 

We have already seen (c.f. (^) and ([70|) ) that 

ISXD = SX + E(-l)'' Set ® 6k. 

But 

{eXD f){x) = e{x - y) f'{y) dy = {Xf){x) + J^i-lf e,(x) /(a,), 

and so 

SXeXD = SX + ^(-1)'^ SXSk ® 4- 

Thus 

ISXD - SXeXD = ^(-1)*^ ^(1 - X)ek ® 4- 

And now, just as at the end of GSE, we conclude that the determinant of ([7^) equals 
£■2(0, J) times the determinant of 

I-T.i-^)'iS + RS)il-X)ek0 5k, 
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which in turn equals the determinant of 

This completes the justification of (^). 
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